Reliability Assessment of Legacy Safety-Critical Systems Upgraded with Fault-Tolerant Off-the-Shelf Software

This paper presents a new way of applying Bayesian assessment to systems, which consist of many components. Full Bayesian inference with such systems is problematic, because it is computationally hard and, far more seriously, one needs to specify a multivariate prior distribution with many counterintuitive dependencies between the probabilities of component failures. The approach taken here is one of decomposition. The system is decomposed into partial views of the systems or part thereof with different degrees of detail and then a mechanism of propagating the knowledge obtained with the more refined views back to the coarser views is applied (recalibration of coarse models). The paper describes the recalibration technique and then evaluates the accuracy of recalibrated models numerically on contrived examples using two techniques: u-plot and prequential likelihood, developed by others for software reliability growth models. The results indicate that the recalibrated predictions are often more accurate than the predictions obtained with the less detailed models, although this is not guaranteed. The techniques used to assess the accuracy of the predictions are accurate enough for one to be able to choose the model giving the most accurate prediction

Design diversity: an update from research on reliability modelling

Diversity between redundant subsystems is, in various forms, a common design approach for improving system dependability. Its value in the case of software-based systems is still controversial. This paper gives an overview of reliability modelling work we carried out in recent projects on design diversity, presented in the context of previous knowledge and practice. These results provide additional insight for decisions in applying diversity and in assessing diverseredundant systems. A general observation is that, just as diversity is a very general design approach, the models of diversity can help conceptual understanding of a range of different situations. We summarise results in the general modelling of common-mode failure, in inference from observed failure data, and in decision-making for diversity in development.

From K.A.M. Tori to Isospectral Invariants and Spectral Rigidity of Billiard Tables

This article is a part of a project investigating the relationship between the dynamics of completely integrable or close to completely integrable billiard tables, the integral geometry on them, and the spectrum of the corresponding Laplace-Beltrami operators. It is concerned with new isospectral invariants and with the spectral rigidity problem for families of Laplace-Beltrami operators with Dirichlet, Neumann or Robin boundary conditions, associated with C^1 families of billiard tables. We introduce a notion of weak isospectrality for such deformations. The main dynamical assumption on the initial billiard table is that the corresponding billiard ball map or an iterate of it has a Kronecker invariant torus with a Diophantine frequency and that the corresponding Birkhoff Normal Form is nondegenerate in Kolmogorov sense. Then we obtain C^1 families of Kronecker tori with Diophantine frequencies. If the family of the Laplace-Beltrami operators satisfies the weak isospectral condition, we prove that the average action on the tori and the Birkhoff Normal Form of the billiard ball maps remain the same along the perturbation. As an application we obtain infinitesimal spectral rigidity for Liouville billiard tables in dimensions two and three. Applications are obtained also for strictly convex billiard tables of dimension two as well as in the case when the initial billiard table admits an elliptic periodic billiard trajectory. Spectral rigidity of billard tables close elliptical billiard tables is obtained. The results are based on a construction of C^1 families of quasi-modes associated with the Kronecker tori and on suitable KAM theorems for C^1 families of Hamiltonians.Comment: 170 pages; new results about the spectral rigidity of elliptical billiard tables; new Modified Iterative Lemma in the proof of KAM theorem with parameter

Assessing Asymmetric Fault-Tolerant Software

The most popular forms of fault tolerance against design faults use "asymmetric" architectures in which a "primary" part performs the computation and a "secondary" part is in charge of detecting errors and performing some kind of error processing and recovery. In contrast, the most studied forms of software fault tolerance are "symmetric" ones, e.g. N-version programming. The latter are often controversial, the former are not. We discuss how to assess the dependability gains achieved by these methods. Substantial difficulties have been shown to exist for symmetric schemes, but we show that the same difficulties affect asymmetric schemes. Indeed, the latter present somewhat subtler problems. In both cases, to predict the dependability of the fault-tolerant system it is not enough to know the dependability of the individual components. We extend to asymmetric architectures the style of probabilistic modeling that has been useful for describing the dependability of "symmetric" architectures, to highlight factors that complicate the assessment. In the light of these models, we finally discuss fault injection approaches to estimating coverage factors. We highlight the limits of what can be predicted and some useful research directions towards clarifying and extending the range of situations in which estimates of coverage of fault tolerance mechanisms can be trusted

Improving DBMS performance through diverse redundancy

Database replication is widely used to improve both fault tolerance and DBMS performance. Non-diverse database replication has a significant limitation - it is effective against crash failures only. Diverse redundancy is an effective mechanism of tolerating a wider range of failures, including many non-crash failures. However it has not been adopted in practice because many see DBMS performance as the main concern. In this paper we show experimental evidence that diverse redundancy (diverse replication) can bring benefits in terms of DBMS performance, too. We report on experimental results with an optimistic architecture built with two diverse DBMSs under a load derived from TPC-C benchmark, which show that a diverse pair performs faster not only than non-diverse pairs but also than the individual copies of the DBMSs used. This result is important because it shows potential for DBMS performance better than anything achievable with the available off-the-shelf servers

Rephrasing rules for off-the-shelf SQL database servers

We have reported previously (Gashi et al., 2004) results of a study with a sample of bug reports from four off-the-shelf SQL servers. We checked whether these bugs caused failures in more than one server. We found that very few bugs caused failures in two servers and none caused failures in more than two. This would suggest a fault-tolerant server built with diverse off-the-shelf servers would be a prudent choice for improving failure detection. To study other aspects of fault tolerance, namely failure diagnosis and state recovery, we have studied the "data diversity" mechanism and we defined a number of SQL rephrasing rules. These rules transform a client sent statement to an additional logically equivalent statement, leading to more results being returned to an adjudicator. These rules therefore help to increase the probability of a correct response being returned to a client and maintain a correct state in the database

Uncertainty explicit assessment of off-the-shelf software: Selection of an optimal diverse pair

Assessment of software COTS components is an essential part of component-based software development. Sub-optimal selection of components may lead to solutions with low quality. The assessment is based on incomplete knowledge about the COTS components themselves and other aspects, which may affect the choice such as the vendor's credentials, etc. We argue in favor of assessment methods in which uncertainty is explicitly represented (`uncertainty explicit' methods) using probability distributions. We have adapted a model (developed elsewhere by Littlewood, B. et al. (2000)) for assessment of a pair of COTS components to take account of the fault (bug) logs that might be available for the COTS components being assessed. We also provide empirical data from a study we have conducted with off-the-shelf database servers, which illustrate the use of the method

The effect of testing on reliability of fault-tolerant software

Previous models have investigated the impact upondiversity - and hence upon the reliability of fault-tolerantsoftware built from 'diverse' versions - of the variation in'difficulty' of demands over the demand space. Thesemodels are essentially static, taking a single snapshotview of the system. In this paper we consider ageneralisation in which the individual versions areallowed to evolve - and their reliability to grow - throughdebugging. In particular, we examine the trade-off thatoccurs in testing between, on the one hand, the increasingreliability of individual versions, and on the other handthe possible diminution of diversity